Performance Evaluation of CLIR and MLIR using Precision Metric Variants

The performance evaluation of information retrieval systems has achieved a high momentum in the last few years. Basic performance measures of information retrieval systems include precision and recall. While these measures work well in monolingual web retrieval, they are not suitable for CLIR (Cross-lingual Information Retrieval) and MLIR (Multilingual Information Retrieval) where two or more languages are involved respectively. Many measures were proposed to improve over the precision-recall measures but they are inadequate to exhibit the language wise performance evaluations. Precision metric variants for evaluating the performance over the retrieval of the documents in various languages have been proposed in this research. This paper also identifies the major strengths and shortcomings of some of the existing IR performance evaluation measures. This paper concentrates on the metric based performance evaluation on two variants of IR. Experiments are conducted in two phases (CLIR and MLIR). These two phases of experiments have been done on practical web search systems and proved that the proposed measures are necessary to reveal the importance of language wise comparisons.